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Section 5 



Bernstein Polynomials. Hausdorff and 
de Finetti theorems. 



Let us look at some applications related to the law of large numbers. Consider an i.i.d. sequence of real 
valued r.v. {Xi) with distribution Fg from a family of distributions parametrized by ^ e 6 C R such that 

EgXi = 9,a^e) := Vare(Xi) <K< +00. 

Let X„ = i Ei<„^i- The following holds. 

Theorem 6 Ifu:W^]S.is uniformly continuous and bounded then E$u{Xn) u{0) uniformly over Q. 
Proof. For any e > 0, 

\Reu{X^) - u{9)\ < Eg\uiX„) - ui9)\ 

= Eg\u{Xn)-u{9)\(l{\X,,-e\<e)+I{\Xn-0\>e)') 

< max \u(x) -u(9)\+2inax\u(x)\F0(\Xn-9\> e) 
\x-e\<£ X 

< 5(£) + 2|hu1e,(x„-^)2<^(£) + ?M^, 

where (5(e) is the modulus of continuity of u. Letting £ = e„ — » so that ne^ 00 finishes the proof. 

□ 

Exeimple. Let [Xi) be i.i.d. with Bernoulli distribution B{ff) with probability of success 9 G [0, 1], i.e. 

Fg{Xi = l) = 9, Fe{Xi = 0) = l-9, 
and let u: [0, 1] — > M be continuous. Then, by the above Theorem, the following Bernstein polynomials 

fe=0 i=l fe=0 

uniformly on [0, 1]. 

Example. Let {X^) have Poisson distribution 11(0) with intensity parameter 9 > defined by 

nk 

Fg{Xi = k) = — e"^ for integer fc > 0. 

Then it is well known (and easy to check) that EgXi = 9, <t'^{9) = 9 and the sum Xi + . . . + X„ has Poisson 
distribution Il{n9). If u is bounded and continuous on [0,+oo) then 



B„(0) ■.= Eeu{X^) = E"(„)P«(E^^ = = ( J^'(l - ^ 



k=0 j=l k=0 
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uniformly on compact sets. 

□ 

Moment problem. Consider a random variable X G [0,1] and let = EX'^ be its moments. Given 
a sequence (cq, ci, C2, . . .) let us define a sequence of increments by Ac/j = Ck+i — Ck- Then 

-A^fc = Hk- l^k+i = - X''+') = EX'=(1 - X), 

{-A){-Ai^k) = {-l)^A^i^k = EX''{1 - X) - EX'=+i(l -X)= EX''{1 - X)^ 
and by induction 

(-l)''A>fc =EX'=(1-X)^ 

Clearly, {—lyA'^^k > since X G [0, 1]. If u is a continuous function on [0, 1] and B„ is its corresponding 
Bernstein polynomial then 

fe=o ^ ' fe=o ^ ' 

Since Bn{X) converges uniformly to u{X), ES„(X) converges to Eu(X). Let us define 

n 

(_l)"-feA"-Vfc > 0, ^pI"^ = 1 (take u = 1). 

fe=0 

We can think of p^"' as the distribution of a r.v. X^") such that 




(5.0.1) 



We showed that 

EB„(X) = Eu(x(")) ^ Eti(X) 

for any continuous function u. We will later see that by definition this means that X^"' converges to X in 
distribution. Given the moments of a r.v. X, this construction allows us to approximate the distribution of 
X and expectation of u{X) . 

□ 

Next, given a sequence (/Ufc), when is it the sequence of moments of some [0, 1] valued r.v. XI By the 
above, it is necessary that 

Mfe > 0, Mo = 1 and (-l)''A>fe > for all fc, r. (5.0.2) 
It turns out that this is also sufficient. 

Theorem 7 (Hausdorff) There exists a r.v. X e [0,1] such that fXk = EX*" iff (5.0.2) holds. 

Proof. The idea of the proof is as follows. If /ij. are the moments of the distribution of some r.v. X, then 
the discrete distributions defined in (5.0.1) should approximate it. Therefore, our goal will be to show that 
condition (5.0.2) ensures that (p^"'') is indeed a distribution and then show that the moments of (5.0.1) 
converge to Hk- As a result, any limit of these distributions will be a candidate for the distribution of X. 

First of all, let us express Hk in terms of (p^."^)- Since A^k = fJ-k+i — M-k we have the following inversion 
formula: 

Mfe = Mfe+i - A/ifc = (/ifc+2 - A/ife+i) + (-A/xfe+i + A^jUfe) 
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by induction. Take r = n — k. Then 

n—k jn—k\ , . n—k (n—k\ 

V .7 / / \/ , Nra-rfe+i-l A n-ffe+i") \ " V .7 ) in) 



li-k = 

J 



We have 



so that 



("7") ^ (»-fc)! {k + j)\{n-k-j)\ ^ {'t') 
Llj) JKn-k-jy. n\ (^) 



n—k (k+j\ n /m\ 

»^ - V - V iAZ„(") 

By (5.0.2), p^;:^ > and E m<nPra — /iQ — 1 SO we Can consider a r.v. X^") such that 

pj^^(n) = ^) = for < m < n. 

We have 

. = V = V ^("^-l)---(^-fc + l) „(n) ^ V 



m— 



Any continuous function u can be approximated by (for example, Bernstein) polynomials so the limit 
lim„_>oo ]Ew(-'i^'"^) exists. By selection theorem that we will prove later in the course, one can choose a 
subsequence X^"*) that converges to some r.v. X in distribution and, as a result, 

which means that /Xfe are the moments of X. 

□ 

de Finetti's theorem. Consider an exchangeable sequence Xi,X2,. ■ ■ ,Xn, ... of Bernoulli random 
variables which means that for any n > 1 the probability 

^{Xr = Xu...,Xn = Xn) 

depends only on a;i + . . . + x„, i.e. it does not depend on the order of I's or O's. Another way to say this is 
that for any n > 1 and any permutation tt of 1, . . . , n the distribution of {X^^l^, ... , X^(„)) does not depend 
on TT. Then the following holds. 

Theorem 8 (de Finetti) There exists a distribution F on [0,1] such that 

Pk := P(Xi + ... + Xn = k) = (j^x'^il- xr-''dF{x). 

This means that to generate such exchangeable sequence we can first pick a; e [0, 1] from distribution F and 
then generate a sequence of i.i.d Bernoulli random variables with probability of success x. 
Proof. Let /xq = 1 and for /c > 1 define 

^ik=nXl = l,...,Xk = l). (5.0.3) 
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We have 

P(Xi = l,...,Xfc = l,Xfe+i = 0) = V{Xi = l,...,Xk = l) 

- ¥{Xi = l,...,Xk = l,Xk+i^l) 

Next, using exchangeability 

P(Xi = l,...,Xfc-l,Xfc+i-0,Xfe+2 = 0) = P(Xi-l,...,Xfe-l,Xfe+i = 0) 

- ¥{Xi = l,...,Xk = l,Xk+i = 0,Xk+2 = l) 
= -A^fc - (-A^fc+i) = A^^fe. 

Similarly, by induction, 

P(Xi = 1, Xfc = 1, = 0, X„ = 0) = (-1)"-'=A"-Vfe > 0. 
By the HausdorfF theorem, iik = EX'^ for some r.v. X e [0, 1] and, therefore, 
P(Xi = l,...,Xfe = l,Xfe+i = 0,...,X„ = 0) = (-1)"-^A"-Vfe 



Jo 



Since, by exchangeability, changing the order of I's and O's does not affect the probabihty, we get 



P(X; 



l + ...+X„ = k)= f 

Jo 



x''{l-xY-^dF{x). 



Example. (Polya urn model). Suppose we have h blue and r red balls in the urn. We pick a ball 



Pick 



• b* 






• 




• 




+ c of tlie same color 



Figure 5.1: Polya urn model, 
randomly and return it with c balls of the same color. Consider r.v.s 

Xj's are not independent but exchangeable. For example, 

h h -\- c T 
br) = X X , V{brb) 



1 if the rth ball picked is blue 
otherwise. 



b r 6+7' 

X X 



fe + r 5 + r + c 5 + r + 2c b + r 6 + r + c b + r + 2c 

are equal. To identify the distribution F in de Finetti's theorem, let us look at its moments /ij, in (5.0.3), 



fik^V{b_^) 
k times 



b b + c b+(k-l)c 

X X • • • X 



b+r " b + r + c" ' b + r+{k-l)c' 
One can recognize or easily check that are the moments of Beta(Q;,/3) distribution with the density 

^^x^-d-x)'^- 
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on [0, 1] with parameters a = b/c,i3 = r/c. By de Finetti's theorem, we can generate Xi's by first picking 
X from distribution Beta(6/c, r/c) and then generating i.i.d. Bernoulh (Xi)'s with probability of success x. 
By strong law of large numbers, the proportion of blue balls in the first n repetitions will converge to this 
probability of success x, i.e. in the limit it will be random with Beta distribution. This example will come 
up once more when we talk about convergence of martingales. 

□ 
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